{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 进程和线程\n",
    "\n",
    "什么叫“多任务”呢？简单地说，就是操作系统可以同时运行多个任务。打个比方，你一边在用浏览器上网，一边在听MP3，一边在用Word赶作业，这就是多任务，至少同时有3个任务正在运行。还有很多任务悄悄地在后台同时运行着，只是桌面上没有显示而已。\n",
    "\n",
    "对于操作系统来说，一个任务就是一个进程（Process），比如打开一个浏览器就是启动一个浏览器进程\n",
    "\n",
    "有些进程还不止同时干一件事，比如Word，它可以同时进行打字、拼写检查、打印等事情。在一个进程内部，要同时干多件事，就需要同时运行多个“子任务”，我们把进程内的这些“子任务”称为线程（Thread）。\n",
    "\n",
    "由于每个进程至少要干一件事，所以，一个进程至少有一个线程。当然，像Word这种复杂的进程可以有多个线程，多个线程可以同时执行\n",
    "\n",
    "我们前面编写的所有的Python程序，都是执行单任务的进程，也就是只有一个线程。如果我们要同时执行多个任务怎么办？\n",
    "\n",
    "\n",
    "### 有两种解决方案：\n",
    "\n",
    "一种是启动多个进程，每个进程虽然只有一个线程，但多个进程可以一块执行多个任务。\n",
    "\n",
    "还有一种方法是启动一个进程，在一个进程内启动多个线程，这样，多个线程也可以一块执行多个任务。\n",
    "\n",
    "同时执行多个任务通常各个任务之间并不是没有关联的，而是需要相互通信和协调，有时，任务1必须暂停等待任务2完成后才能继续执行，有时，任务3和任务4又不能同时执行，所以，多进程和多线程的程序的复杂度要远远高于我们前面写的单进程单线程的程序。\n",
    "\n",
    "多进程和多线程的程序涉及到同步、数据共享的问题，编写起来更复杂。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多进程\n",
    "\n",
    "要让Python程序实现多进程（multiprocessing），我们先了解操作系统的相关知识。\n",
    "\n",
    "Unix/Linux操作系统提供了一个fork()系统调用，它非常特殊。普通的函数调用，调用一次，返回一次，但是fork()调用一次，返回两次，因为操作系统自动把当前进程（称为父进程）复制了一份（称为子进程），然后，分别在父进程和子进程内返回。\n",
    "\n",
    "子进程永远返回0，而父进程返回子进程的ID。这样做的理由是，一个父进程可以fork出很多子进程，所以，父进程要记下每个子进程的ID，而子进程只需要调用getppid()就可以拿到父进程的ID。\n",
    "\n",
    "Python的os模块封装了常见的系统调用，其中就包括fork，可以在Python程序中轻松创建子进程："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "print('进程 (%s) 启动...' % os.getpid())\n",
    "# Only works on Unix/Linux/Mac:\n",
    "pid = os.fork()\n",
    "\n",
    "if pid == 0:\n",
    "    print('我是子进程(%s) ，我的父进程是(%s).' % (os.getpid(), os.getppid()))\n",
    "else:\n",
    "    print('我 (%s) 创造了一个子进程 (%s).' % (os.getpid(), pid))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 由于Windows没有fork调用，上面的代码在Windows上无法运行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### multiprocessing\n",
    "如果你打算编写多进程的服务程序，Unix/Linux无疑是正确的选择。由于Windows没有fork调用，难道在Windows上无法用Python编写多进程的程序？\n",
    "\n",
    "由于Python是跨平台的，自然也应该提供一个跨平台的多进程支持。multiprocessing模块就是跨平台版本的多进程模块。\n",
    "\n",
    "multiprocessing模块提供了一个Process类来代表一个进程对象，下面的例子演示了启动一个子进程并等待其结束："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from multiprocessing import Process\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 子进程要执行的代码\n",
    "def run_proc(name):\n",
    "    print('运行子进程 %s (%s)...' % (name, os.getpid()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print('父进程 %s.' % os.getpid())\n",
    "p = Process(target=run_proc, args=('test',))\n",
    "print('开启父进程.')\n",
    "p.start()\n",
    "p.join()\n",
    "print('子进程结束.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "创建子进程时，只需要传入一个执行函数和函数的参数，创建一个Process实例，用start()方法启动，这样创建进程比fork()还要简单。\n",
    "\n",
    "join()方法可以等待子进程结束后再继续往下运行，通常用于进程间的同步。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 进程间通信\n",
    "\n",
    "Process之间肯定是需要通信的，操作系统提供了很多机制来实现进程间的通信。Python的multiprocessing模块包装了底层的机制，提供了Queue、Pipes等多种方式来交换数据。\n",
    "\n",
    "我们以Queue为例，在父进程中创建两个子进程，一个往Queue里写数据，一个从Queue里读数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from multiprocessing import Process, Queue\n",
    "import os, time, random\n",
    "\n",
    "# 写数据进程执行的代码:\n",
    "def write(q):\n",
    "    print('Process to write: %s' % os.getpid())\n",
    "    for value in ['A', 'B', 'C']:\n",
    "        print('Put %s to queue...' % value)\n",
    "        q.put(value)\n",
    "        time.sleep(random.random())\n",
    "\n",
    "# 读数据进程执行的代码:\n",
    "def read(q):\n",
    "    print('Process to read: %s' % os.getpid())\n",
    "    while True:\n",
    "        value = q.get(True)\n",
    "        print('Get %s from queue.' % value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 父进程创建Queue，并传给各个子进程：\n",
    "q = Queue()\n",
    "pw = Process(target=write, args=(q,))\n",
    "pr = Process(target=read, args=(q,))\n",
    "# 启动子进程pw，写入:\n",
    "pw.start()\n",
    "# 启动子进程pr，读取:\n",
    "pr.start()\n",
    "# 等待pw结束:\n",
    "pw.join()\n",
    "# pr进程里是死循环，无法等待其结束，只能强行终止:\n",
    "pr.terminate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多线程\n",
    "\n",
    "多任务可以由多进程完成，也可以由一个进程内的多线程完成。\n",
    "\n",
    "我们前面提到了进程是由若干线程组成的，一个进程至少有一个线程。\n",
    "\n",
    "由于线程是操作系统直接支持的执行单元，因此，高级语言通常都内置多线程的支持，Python也不例外，并且，Python的线程是真正的Posix Thread，而不是模拟出来的线程。\n",
    "\n",
    "Python的标准库提供了两个模块：_thread和threading，_thread是低级模块，threading是高级模块，对_thread进行了封装。绝大多数情况下，我们只需要使用threading这个高级模块。\n",
    "\n",
    "启动一个线程就是把一个函数传入并创建Thread实例，然后调用start()开始执行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import time, threading, random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 新线程执行的代码:\n",
    "def loop():\n",
    "    print('线程 %s 运行中...' % threading.current_thread().name)\n",
    "    n = 0\n",
    "    while n < 5:\n",
    "        n = n + 1\n",
    "        print('线程 %s >>> %s' % (threading.current_thread().name, n))\n",
    "        time.sleep(1)\n",
    "    print('线程 %s 结束.' % threading.current_thread().name)\n",
    "\n",
    "print('线程 %s 在运行...' % threading.current_thread().name)\n",
    "t = threading.Thread(target=loop, name='LoopThread')\n",
    "t.start()\n",
    "t.join()\n",
    "print('线程 %s 结束.' % threading.current_thread().name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于任何进程默认就会启动一个线程，我们把该线程称为主线程，主线程又可以启动新的线程，Python的threading模块有个current_thread()函数，它永远返回当前线程的实例。主线程实例的名字叫MainThread，子线程的名字在创建时指定，我们用LoopThread命名子线程。名字仅仅在打印时用来显示，完全没有其他意义，如果不起名字Python就自动给线程命名为Thread-1，Thread-2……\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 应用例子\n",
    "\n",
    "多线程爬虫，用多线程提高爬虫效率，爬虫花费大量时间在等待网页的回复，CPU利用率不高，所以我们可以同时打开多个网页，提高效率！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import threading\n",
    "import requests\n",
    "import time,random\n",
    "\n",
    "\n",
    "def func1(url):\n",
    "    print('打开网页%s, 模拟爬虫工作' % url)\n",
    "    res = requests.get(url)\n",
    "    time.sleep(random.randint(0,30))\n",
    "    print('结束，%s 返回结果 %s' % (url, res.status_code))\n",
    "\n",
    "def func2(urlinfo):\n",
    "    for i in urlinfo:\n",
    "        th = threading.Thread(target=func1,args=[i])\n",
    "        th.start()\n",
    "    print('主程序结束')\n",
    "\n",
    "\n",
    "urlinfo = ['http://www.sohu.com', 'http://www.163.com', 'http://www.sina.com']\n",
    "func2(urlinfo)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 练习 输出 0到11，观察他们的输出顺序\n",
    "import threading\n",
    "num = 0\n",
    "\n",
    "\n",
    "def t():\n",
    "    global ???\n",
    "    num += 1\n",
    "    print(num)\n",
    "\n",
    "for i in range(0, 11):\n",
    "    d = threading.Thread(???)\n",
    "    d.???"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
